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ABSTRACT 

Educational researchers find mail questionnaires a 
convenient and relatively inexpensive way to conduct a survey, but 
response rate remains a crucial issue. This research examines the 
effects of two manipulated variables, the appeal of the introductory 
letter and a nonmonetary incentive, on survey response rate. Gender 
was also used as an explanatory variable. Subjects were full-time 
professors. The instrument addressed faculty attitudes about the 
importance of scholarly activities. The nonmonetary incentive was a 
decaffeinated coffee bag. Cover letter appeal was more personal in 
the experimental letter than in the control letter. Of the 950 
deliverable surveys, 422 subjects responded to the initial mailing, 
with 552 subjects eventually responding after 2 followup postcard 
reminders. The hypothesis that nonmonetary incentives increase 
response rate was supported; incentives increased the initial 
response rate slightly, at 8 . 8 % for those receiving the coffee. There 
was no significant effect of the cover letter appeal or of gender. 
Benefits of the small increase in initial response must be weighed 
against the increased cost of conducting the survey. Two tables 
present survey findings. (Contains 17 references.) (SLD) 
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Educational researchers utilize iru il questionnaires since they are 
convenient and relatively inexpensive compared to interviews or ethnographic 
research. In addition, a greater number of subjects may be obtained using 
survey research, thereby facilitating the estimation of population parameters 
from sample statistics. Response rate remains a crucial issue if randomness of 
the sample is to be maintained. An inadequate response rate may introduce 
bias into the results of the research. The purpose of this research was to 
examine the effects of two manipulated variables, introductory appeal letter 
and a nonmonetary incentive, on response rate. Gender was also used as an 
explanatory variable. 

Questionnaires are utilized often in research regarding college faculty 
(Yuker, 1984). National faculty surveys report response rates ranging from 36% 
to 62% (Braxton, 1983; Gray, Froh, & Diamond, 1992; McGee & Ford, 1987). 
These response rates are considered typical for national faculty surveys (Locke, 
Fitzpatrick, & White, 1983), but it is desirable to increase these response rates 
to maintain sample representation. Small monetary incentives (twenty-five 
cents to one dollar) have been found increase response rates in the general 
population (Brennan, 1992; Gajraj, Faria & Dickinson, 1988; Goyder, 1982; 
Yammarino, Skinner & Childers, 1991). Nonmonetary incentives have not been 
investigated as a means to increase response rate. 

The. effects of varying cover letter appeals have been previously 
investigated with contradictory results (Green, Jacobi, Lam, Boser & Hall, 
1993). Appeals that have been manipulated include social utility of responses 
(utility) and benefits to the sponsor (sponsor). A social utility appeal stresses 
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the importance of response to the betterment of society whereas a sponsor 
appeal emphasizes importance in aiding the sender. Results of appeal research 
have been mixed. Studies have found a positive significant effect of social 
utility over sponsor appeal (Jones & Linda, 1978); a negative effect of social 
utility versus sponsor appeal (Biner, 1988; Childers, Pride, & Ferrell, 1980); and 
no significant effect of appeal type (Linsky, 1965). 

Research regarding gender differences in response rates is mixed. 
Goyder (1982) indicates that males respond more frequently than females. 
Green and Stager (1986) report that males respond to an initial mailing at a 
significantly higher rate. Female teachers tend to respond at a significantly 
higher rate than male teachers (Green, Jacobi, Lam, Boser & Hall, 1993). Y^t, 
Green and Kvidahl (1989) suggest that males and females do not differ in 
response frequency. 

The purpose of this research was to test the following hypotheses: 

1. response rates are significantly higher if a nonmonetary incentive is 
offered than if no incentive is offered; 

2. social utility appeal will significantly increase response rate above 
sponsor appeal; 

3. females and males will significantly differ in response rate. 
Method 

Subjects were 1,000 full-time professors at doctorate-granting or 
comprehensive universities. One hundred institutions were selected using a 
stratified random sampling technique. The two stratification variables were 
institutional type (public or private) and Carnegie classification: Doctorate- 
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Granting I, Doctorate-Granting II, Comprehensive I, and Comprehensive II. 
College catalogs provided the population list. The sample was stratified by 
gender (equal representation from males and females), department (equal 
representation from eight departmental categories), and rank (assistant 
professor, associate professors, and full professor in a ratio of 4:3:3). 

The five page, 104 item instrument addressed faculty attitudes about the 
personal importance of scholarly activities, institutional importance of 
scholarly activities, amount of time spent on scholarly and recreational 
activities, number of publications, attitudes about faculty workload, and 
demographic data. Attitude questions utilized a five point rating scale. 
Questions regarding time offered five choices for response. Items regarding 
publications were free response. All but one demographic item offered 
response choices. All surveys were personalized both in the salutation and in 
hand signature. All subjects were offered a summary of the results of the 
survey. 

Since monetary incentives have been researched, a nonmonetary incentive 
was chosen. The nonmonetary incentive was a one-cup, sealed, decaffeinated 
coffee bag. A decaffeinated, sealed beverage was chosen because of the 
researcher's perception of health trends. Decaffeinated coffee in a sealed bag 
is relatively inexpensive (approximately $ 0.50 per bag) and readily available. 
Subjects were randomly assigned to an incentive or no incentive condition. 
Appeal to the subject was achieved with the following sentence in the cover 
letter "Your input into this matter is very important in determining what 
faculty consider scholarship to be." Subjects who did not receive this appeal 



had the following sentence in the cover letter "It is important to ascertain 
what faculty consider scholarship to be in order to develop models of 
scholarship and further knowledge in this field." An equal number of subjects 
received each type of appeal. In the cover letter, subjects were asked to 
return the survey blank if they did not wish to participate. 

Surveys were coded in the upper right hand corner of the first page to 
identify nonrespondents for follow-up mailings. The surveys were mailed in 
April. The first mailing included a cover letter, the survey, a stamped, self- 
addressed envelope, and an index card. Subjects indicated their desire to 
receive a summary of results by returning the index card with their name and 
address. In addition, 500 of the subjects received a coffee bag in the first 
mailing. The first follow-up was a postcard reminder. It was sent 
approximately three weeks after the initial mailing. The second follow-up was 
a cover letter, a second copy of the survey, and a stamped, addressed return 
envelope. It was mailed approximately three weeks after the postcard 
reminder. Forty-one of the surveys were returned as undeliverable; subjects 
had moved to another institution, were ill or deceased, were on sabbatical, or 
had retired. Nine of the subjects responded that the survey was inappropriate 
for them since they were not full-time or were in administration. The 
effective sample size was 950. 

Data were analyzed using a hierarchical logit model (Kennedy, 1992) to 
assess the effects of the nonmonetary incentive, the appeal, and gender on 
response rate. Three analyses were performed. All three were 2x2x2 
factorial designs using incentive, appeal, and gender as explanatory variables 
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and response as the dependent variable. In the first analysis, no response and 
refusals were combined to create a dichotomous dependent variable (completed 
/ refused or not returned). The second analysis used response as a three level 
variable, considering the categories completed survey, refusal, and nonresponse. 
The third analysis considered only subjects who had responded to the initial 
mailing, to determine the effects of the independent variables on initial 
response. Response was the dependent variable with two levels, completed and 
refused. 
Results 

Of the 950 deliverable surveys, 552 subjects completed the survey. One 
hundred twenty-three returned the survey unanswered. The survey return 
rate was 71.5%; the completion rate was 58%. Four hundred twenty-two 
subjects responded to the initial mailing (354 complete, 68 refusals), 109 to the 
postcard follow-up (85 complete, 24 refusals), and 144 to the second follow-up 
( 113 complete, 31 refusals). 

For the first two analyses, 931 subjects were used. The gender of 19 

subjects was unknown since the college catalog listed only initials for first 

name or names of the faculty member were not identifiable as male or female. 

2 

A significant main effect was found for the incentive (residual L =1.75, p=.780; 
component L^=6.78; p< 0.01). No other variables attained significant component 

values (see Table 1). Of the subjects receiving coffee, 63.5% returned a 
completed survey. Of those who did not receive coffee, 54.7% returned the 
survey complete. Yule's Q between incentive and response was 0.19. 
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Table 1 

RpgiHnai and C.-r.npone.nt Ass ociations for Logit M o del Effe cts 



Marginals Residual 






Component 




Fit 


L 2 


df 


P 


L 2 df 


P 


ABC.D. 


10.21 


7 


.177 






ABC.AD. 


9.24 


6 


.160 


0.971 1 


.50 


ABQAD.BD. 


8.53 


5 


.130 


0.711 1 


.50 


ABC,AD,BD,CD. 


1.75 


4 


.780 


6.781 1 


.01 


ABC,ABD,CD. 


1.66 


3 


.647 


0.091 1 


>.50 


ABC,ABD,ACD. 


0.31 


2 


.857 


1.351 1 


.25 


ABC,ABD,ACD,BCD. 


0.26 


1 


.611 


0.051 1 


>.50 


ABCD. 


0.00 


0 


1 


0.261 1 


>.50 



A=Appeal B=Gender C=Incentive D=Response 

In the second analysis, the dependent variable had three levels: returned 
complete, refused, or no response. Again, incentive provided a significant 
effect (residual L 2 =6.16, p=.629; component L 2 =6.78; p< 0.01). Subjects 
receiving no incentive were more likely to return a blank survey or none at 
all (See Table 2). 
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Table 2 



Incentive Effect on Response Rate 





Complete 


Refused 


No Response 


Total 


Incentive 


287 


53 


112 


452 


Row % 


63.5% 


11.7% 


24.8% 




Column% 


51.2% 


43.4% 


43.1% 




No Incentive 


262 


69 


148 


479 


Row % 


54.7% 


14.4% 


30.9% 




Column % 


49.8% 


66.6% 


66.9% 




Total 


552 


122 


260 


931 



The third analysis was restricted to those who responded to the initial 
mailing. The dependent variable was restricted to two levels, responded or 
refused (returned the survey blank). There was a significant main effect for 
incentive (residual L 2 =1.96; p=.743; component L 2 =5.49; p< 0.02) with a 4.8% 
higher completed response rate for those who received an incentive. Those 
who received no incentive had a 5.6% higher refusal rate. 
Discussion 

The hypothesis that nonmonetary incentives increase response rate was 
supported, and the incentives served to increase slightly (4.8%) the response 
rate to the initial mailing. There was an increase of 8.8% in overall return 
rate for those who received coffee over those who did not. The second 
hypothesis was not confirmed since there was no significant difference in 
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response rate between those who received a utility versus a sponsor appeal. 
This may be due to the strength of the appeal The appeal was contained in a 
single sentence. Researchers studying appeal may consider varying the 
strength of the appeal to study its effect. A multiple sentence appeal may 
contribute to an increased response rate. No significant response rate 
differences between males and females were found, contradicting research of 
Goyder (1982), Green and Stager (1986), Green, Jacobi, Lan, Bosor, and Hall 
(1993) but confirming the research results of Green and Kvidahal (1989). This 
may suggest that male and female university faculty do not respond at 
different rates; previous research finding differing male/female response rates 
has focused on other populations. 

The cost of including the incentive was approximately .75 per survey 
(cost of incentive plus extra postage). Benefits of the small increase in initial 
response must be weighed against costs of the incentive. A similar increase in 
response rate may be gained by choosing to send a third follow-up rather than 
including the incentive in the initial mailing. Subjects who received an 
incentive were less likely to refuse to complete the survey (i.e. return a blank 
survey). It is possible that including the statement about returning the blank 
survey made it easier for subjects to refuse to participate. Future studies may 
consider using that statement as a manipulated variable. 

This population was restricted to college faculty. Different results may 
be obtained if a different population were targeted. 
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